Jon Williamson Foundations for Bayesian Networks

Bayesian networks are normally given one of two types of foundations: they are either treated purely formally as an abstract way of representing probability functions , or they are interpreted, with some causal interpretation given to the graph in a network and some standard interpretation of probability given to the probabilities specified in the network. In this chapter I argue that current foundations are problematic, and put forward new foundations which involve aspects of both the interpreted and the formal approaches. One standard approach is to interpret a Bayesian network objectively: the graph in a Bayesian network represents causality in the world and the specified probabilities are objective, empirical probabilities. Such an interpretation founders when the Bayesian network independence assumption (often called the causal Markov condition) fails to hold. In Ü2 I catalogue the occasions when the independence assumption fails, and show that such failures are pervasive. Next, in Ü3, I show that even where the independence assumption does hold objectively, an agent's causal knowledge is unlikely to satisfy the assumption with respect to her subjective probabilities, and that slight differences between an agent's subjective Bayesian network and an objective Bayesian network can lead to large differences between probability distributions determined by these networks. To overcome these difficulties I put forward logical Bayesian foundations in Ü5. I show that if the graph and probability specification in a Bayesian network are thought of as an agent's background knowledge, then the agent is most rational if she adopts the probability distribution determined by the Bayesian network as her belief function. Specifically, I argue that causal knowledge constrains rational belief via what I call the causal irrelevance condition, and I show that the distribution determined by the Bayesian network maximises entropy given the causal and probabilistic knowledge in the Bayesian network. Now even though the distribution determined by the Bayesian network may be most rational from a logical point of view, it may not be close enough to objective probability for practical purposes. I show in Ü6 that by adding arrows to the Bayesian network according to a conditional mutual information arrow weight-ing, one can decrease the cross entropy distance between the Bayesian network distribution and the objective distribution. This can be done within the context of constraints on the Bayesian network which limit its size and the time taken to calculate probabilities from the network, in order to minimise computational complexity. This …